CI: use 12 groups to run Python UTs #5131
Conversation
Signed-off-by: Jinzhe Zeng <[email protected]>
There was a problem hiding this comment.
Pull request overview
This PR increases the parallelization of Python unit tests in CI by doubling the number of test groups from 6 to 12. This change should help reduce overall CI execution time by distributing the test workload across more parallel jobs.
Key Changes
- Matrix strategy expanded from 6 to 12 groups for parallel test execution
- Pytest splits configuration updated to match the new group count
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
📝 WalkthroughWalkthroughThe GitHub Actions Python test workflow is expanded to increase test parallelism. The job matrix group increases from 6 to 12 entries, and pytest splits double from 6 to 12 accordingly. Duration caching keys are affected by the new splits count, though the cache key structure remains tied to python and group variables. Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Possibly related PRs
Suggested reviewers
Pre-merge checks✅ Passed checks (3 passed)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 Gitleaks (8.30.0).github/workflows/test_python.ymlpanic: allocator_unix: failed to reserve memory: cannot allocate memory goroutine 1 [running]: ... [truncated 302 characters] ... 1e0, {0xf8a300, 0xc000242a80}, 0x0?, {0xe18d1d?, 0x0?}, 0x0?, {0x1932260, 0x0, 0x0}) Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
.github/workflows/test_python.yml (1)
60-60: LGTM! Pytest splits correctly updated.The
--splits 12parameter correctly matches the new matrix group count. The--splitting-algorithm least_durationcombined with duration caching should help distribute tests evenly across the 12 groups.Consider monitoring the actual test distribution and execution time across groups after this change to ensure the parallelism improvement is optimal.
📜 Review details
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/test_python.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (40)
- GitHub Check: Analyze (python)
- GitHub Check: Test Python (12, 3.13)
- GitHub Check: Test Python (11, 3.13)
- GitHub Check: Analyze (c-cpp)
- GitHub Check: Test Python (9, 3.10)
- GitHub Check: Test Python (9, 3.13)
- GitHub Check: Test Python (2, 3.13)
- GitHub Check: Test Python (6, 3.13)
- GitHub Check: Test Python (7, 3.13)
- GitHub Check: Test Python (10, 3.10)
- GitHub Check: Test Python (4, 3.13)
- GitHub Check: Test Python (8, 3.13)
- GitHub Check: Test Python (1, 3.10)
- GitHub Check: Test Python (10, 3.13)
- GitHub Check: Test Python (6, 3.10)
- GitHub Check: Test Python (3, 3.13)
- GitHub Check: Test Python (8, 3.10)
- GitHub Check: Test Python (7, 3.10)
- GitHub Check: Test Python (5, 3.10)
- GitHub Check: Test Python (12, 3.10)
- GitHub Check: Test Python (4, 3.10)
- GitHub Check: Test Python (5, 3.13)
- GitHub Check: Test Python (11, 3.10)
- GitHub Check: Test Python (3, 3.10)
- GitHub Check: Test Python (1, 3.13)
- GitHub Check: Test Python (2, 3.10)
- GitHub Check: Build wheels for cp310-manylinux_aarch64
- GitHub Check: Build C library (2.18, libdeepmd_c.tar.gz)
- GitHub Check: Build wheels for cp311-macosx_x86_64
- GitHub Check: Build wheels for cp311-win_amd64
- GitHub Check: Build wheels for cp311-macosx_arm64
- GitHub Check: Build wheels for cp311-manylinux_x86_64
- GitHub Check: Test C++ (false, false, false, true)
- GitHub Check: Test C++ (false, true, true, false)
- GitHub Check: Test C++ (true, false, false, true)
- GitHub Check: Test C++ (true, true, true, false)
- GitHub Check: Build C++ (clang, clang)
- GitHub Check: Build C++ (cpu, cpu)
- GitHub Check: Build C++ (rocm, rocm)
- GitHub Check: Build C++ (cuda120, cuda)
🔇 Additional comments (1)
.github/workflows/test_python.yml (1)
21-21: LGTM! Doubling test parallelism.The matrix expansion from 6 to 12 groups effectively doubles the test parallelism, which should significantly reduce overall CI runtime. The change is consistent with the corresponding pytest splits parameter update.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #5131 +/- ##
==========================================
- Coverage 82.15% 82.14% -0.01%
==========================================
Files 709 709
Lines 72468 72470 +2
Branches 3616 3615 -1
==========================================
- Hits 59535 59531 -4
- Misses 11769 11773 +4
- Partials 1164 1166 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
2 participants
Fix CI errors.
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.